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Device for Carrying Out Medical Interventions an d a Method f or 

Generating an Image 
^ < 

The invention relates to a device for carrying out medical 
interventions (operations) on the human or animal body, comprising 
a device for storing first image data of the area to be operated on 
which are, for example, obtained by means of computer tomography 

r| and/or magnetic resonance imaging, and comprising a device for 

Ui displaying image data. 

S| In contrast to operations in the orthopedic field, the area to be 
operated on, for example, in brain surgery and liver surgery, will 

^: change during the intervention. In the case of, for example, 
coagulation of a brain tumor there are actually two effects. 

[=1 
\l 

U The brain tumor, on the one hand, will drop, and, on the other 

z: hand, will decrease its volume, wherein the brain will fill out 

ui 

hi immediately the volume that has been freed by the coagulating 
tumor. Accordingly, it is required to intraoperatively update pre- 
operatively obtained image data in order to prevent that, as a 
result of false or missing information in regard to the actual 
position of the tumor to be coagulated, damage will be caused to 
the adjoining brain. 

In order to obtain detailed information in regard to the area to be 
operated on, image -providing methods are used which are, for 
example, magnetic resonance imaging and computer tomography. Both 
image -generating methods are suitable only to a limited extent, or 
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not at all, for intraoperative image generation. Image -generating 
methods by means of ultrasound can be used intraoperatively , but 
the image data obtained therewith are usually not detailed enough. 

It is therefore an object of the invention to provide a device of 
the aforementioned kind with which, intraoperatively, sufficiently 
detailed image data can be obtained and displayed. 

According to the invention the above object is solved for a device 
of the aforementioned kind by an image -generating device, which, 
for example, operates with ultrasound, for obtaining second image 
data of the area to be operated on during the operation and an 
updating device which is configured to compare second image data, 
obtained at a first point in time, with second image data, obtained 
at a second point in time that is after the first point in time, 
and to update the first image data according to the change 
resulting from the comparison, and to supply the updated first 
image data to the display device. 

In other words, image data, which cannot be obtained or obtained 
only with difficulty intraoperatively, are however obtained 
preoperatively with sufficient detail and can be updated and 
displayed intraoperatively by means of image data which can be 
obtained more easily but with less detail during operation so that 
an updated and sufficiently detailed, even though artificially 
generated, image is obtained and displayed. Accordingly, according 
to the invention the image data obtained intraoperatively are not 
themselves displayed but are only used to update substantially more 
detailed image data obtained preoperatively. 

Methods for updating image data are generally known in medical 
technology, for example, from the following publications: 



"Individualizing Anatomical Atlases of the Head" in "Visualization 
in Biomedical Computing", 1996/ "Medical Image Segmentation Using 
Topologically Adaptable Surfaces", in "CVRMed-MRCAS '97", March 
1997, pp. 23-32; "Segmentation Using Deformable Models with 
Affinity-Based Localization", in "CVRMed-MRCAS '97", March 1997, 
pp. 54-62; "Volumetric Medical Images Segmentation Using Shape 
Constrained Deformable Models", in "CVRMed-MRCAS '97", March 1997, 
pp. 13-22; "Shape-Based Segmentation and Tracking in 4D Cardiac MR 
Images", in "CVRMed-MRCAS '97", March 1997, pp. 43-52; "Decimation 
of Iso-Surfaces with Deformable Models", in "CVRMed-MRCAS '97", 
March 1997, pp. 84-92, 

According to the invention it can be preferably provided that the 
updating device updates the first image data in certain time 
intervals. The time intervals are adjusted in this context to the 
specific requirements of the operation to be performed. Some 
changes, such as, for example, shrinkage in connection with 
coagulation, require that updating is perfomed essentially in real 
time. This can mean many updates per second. 

However, it may also be provided that the updating device updates 
the first image data every time when the changes resulting from the 
comparison surpass a certain limit. In other words, an update is 
carried out only when this is required because changes that have to 
be considered have occurred. In this way, the computing time and 
computing capacity required for the update can be reduced to a 
minimum. 

Preferably, the device according to the invention has a surgical 
robot which is configured to carry out, taking into account the 
updated first image data, manually input commands and/or to 
automatically perform at least one operation step according to a 




predetermined program. In other words, the surgical robot works 
automatically based on the respectively updated first image data 
and/or it is guided by a surgeon who can take the respectively 
updated image data from the display device and who has optionally 
direct visual contact with the area to be operated on by means of 
endoscopic devices. 

According to the invention it can be provided that the surgical 
robot is configured to work within the limits of a predetermined 
volume within the area to be operated on and represented by the 
updated first image data. This volume can be, for example, a brain 
tumor to be coagulated. Accordingly, the surgical robot in this 
case is limited to the coagulation of the tumor so that the 
adjoining brain substance is not endangered. 

According to the invention, the surgical robot be configured to 
maintain a predetermined spacing to a predetermined boundary 
surface in the area to be operated on and represented by the 
updated first image data. In this connection it holds true also 
that the brain substance bordering the area to be operated on is 
protected against damage by the surgical robot. 

Moreover, according to the invention, the surgical robot can be 
configured to move to a predetermined point in the area to be 
operated on and represented by the updated first image data. Such 
a target point can be, for example, the center of a brain tumor to 
be coagulated. 

According to the invention, it is preferred to provide a device for 
inputting the limit volume, the boundary surface and/or the target 
point. By doing so, the surgeon has the possibility to set 
preoperatively and/or intraoperatively the limits for the surgical 




robot required for safety and described above or to define a target 
point which is to be reached by the surgical robot at the 
beginning, during the course, or at the end of the operation. 

According to the invention, it is preferred to provide a 
calibrating device which is to be arranged fixedly on the body and 
which has at least one landmark which, relative to the body, 
represents a fixed conimon reference point for the first and the 
second image data. Such a calibrating device serves, for example, 
for enabling adjusting, directly before the beginning of the 
operation, image data, obtained by ultrasound, with image data 
obtained preoperatively by way of computer tomography or by 
magnetic resonance imaging by means of so-called co- registration. 

The calibrating device, according to the invention, is preferably 
formed by a stereotactic frame. Of course, other (equivalent) 
devices can be used for this purpose. 

The invention concerns moreover a method for generating an image of 
the human or animal body. 

As has been mentioned above, there are situations, for example, 
during an operation, in which only such image generating methods 
can be used which provide insufficiently detailed image data. 

The invention also concerns the object to provide a method of the 
aforementioned kind which provides intraoperatively detailed image 
data even when a correspondingly detailed image generating method 
cannot be used intraoperatively. 

According to the invention, the above object is solved by a method 
with the following steps : 
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storing first image data of the body obtained, for example, by 
means of computer tomography or magnetic resonance imaging, 

recording second image data of the body, for example, by means 
of ultrasound, at a first point in time and a second point in 
time which is after the first point in time; 

comparing with one another the second image data recorded at 
the first and the second points in time; 

updating the first image data according to the change 
resulting from the comparison; and 

displaying the updated first image data. 

As has been mentioned above in detail with reference to the device 
according to the invention, according to the method of the 
invention in certain situations, i.e., for example, during an 
operation, an image generating method is used which provides less 
detailed image data, wherein it is not these less detailed image 
data but the detailed image data updated based on the change of the 
less detailed image data that are displayed. 

It is expressly stated that the method according to the invention 
relates exclusively to the generation or display of an image of the 
human or animal body and not to a diagnostic method or therapy. 
For a diagnostic method it is indeed required not only to generate 
and display the image but also to evaluate it. For a therapy, 
measures on the body are required which surpass the pure generation 
and the pure display. 
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According to the invention, the first point in time and the 
position of the body are selected upon recording of the second 
image data such that the second image data taken at the first point 
in time correspond to the stored first image data. 

In other words, a calibration is performed inasmuch as the first 
and the second image data are adjusted once relative to one another 
in that they represent one and the same state of the body. 

According to the invention, the step of updating can be performed 
in predetermined time intervals. 

However, it can also be provided that the step of updating is 
performed (only) when the changes resulting from the comparison 
surpass a predetermined limit. This makes it possible to save 
computing time and computing capacity. 

In the following, the invention is explained in more detail with 
the aid of preferred embodiments with further details with 
reference to the attached drawing, wherein the drawing 

shows schematically an embodiment of the device according to 
the invention. 

The core piece of the device according to the invention is a 
computer 10 with a comparator 12 . By means of a stereotactic frame 
14 fixedly connected on the body, for example, connected to the 
head for neurosurgical purposes, a computer tomogram and/or a 
magnetic resonance image is recorded preoperatively, with 
consideration of landmarks, by means of a computer tomographic or 
a magnetic resonance imaging device 16 . The obtained image data 
are then stored in a memory 18. At the beginning of the operation, 



by using the same stereotactic frame 14, an ultrasound image is 
recorded by means of an ultrasound device 20 which is then adjusted 
for calibration purposes by means of the computer tomogram or the 
magnetic resonance image. 

The preoperatively obtained computer tomogram or magnetic resonance 
image is displayed by means of the display, for example, in the 
form of a monitor 22, 

During the course of the operation, which is performed, for 
example, by means of a surgical robot 24, ultrasound images are 
produced again and again which are compared with the preceding 
ultrasound images. The deviations which are obtained thereby serve 
for changing the computer tomogram or the magnetic image 
accordingly. The- most recent version of the computer tomogram or 
the magnetic resonance image which has been updated accordingly is 
supplied, on the one hand, from the computer 12 to the monitor 22 
so that the surgeon is informed continuously about the course of 
the operation. On the other hand, it is also used in order to 
control the surgical robot 24, wherein this includes, for example, 
that it is being input with a volume to be operated (changing 
during the course of the operation) , wherein it is furthermore also 
programmed to maintain a predetermined spacing from a boundary 
surface previously input into the computer. 

The surgical robot 24, inter alia, is capable of the following: 

automatic limitation to a space defined by the surgeon; 

automatically maintaining a predetermined spacing to a surface 
in the area operated and/or. 
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return to a preselected point in the area to be operated on or 
along the operative access path. 

Programming of this robotic performance is realized in that the 
surgeon provides manually limit values on slice images of pre- 
operatively obtained image data sets which then result in points, 
distances or spaces and can be used for the control of the surgical 
robot 24. Such a point can be a starting or target point or a 
return point on an operative access path, A distance can be a 
connecting line between different points on an access path to an 
area to be operated on or within the area to be operated on. A 
space or volume can be the periphery of a solid or cystic tumor or 
a cyst or its wall whose outer boundary is not to be touched by a 
surgical instrument . 

For controlling the spatial conditions and their changes during an 
operation, the surgeon uses the intraoperative ultrasound imaging. 

Before the operation, a 3-D data set is generated with an image- 
forming method such as magnetic resonance imaging or computer 
tomography. This data set serves as a reference image for the 
further steps. During this examination, the stereotactic frame 14 
is fastened on the head of the patient for neurosurgical purposes. 
This frame 14 contains landmarks which can be used for calibration 
or CO- registration of the systems at the beginning of the 
operation. At the point of beginning the operation, the surgical 
robot 24, which is, for example, in the form of a robot arm, is 
brought into a fixed predetermined spatial relation to the mounted 
stereotactic frame 14 and is fixedly mounted. Subsequently, the 
landmark points are traced with the tip of the operating instrximent 
functioning as the calibration device so that an unequivocal 
relation between the working space of the robot and the space of 
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the preoperatively produced images is generated. A similar fixed 
correlation is then generated between the stereotactic frame 14 and 
the ultrasound head 20 to be implanted for generating three- 
dimensional ultrasound images. With this fixed predetermined 
correlation between the preoperative image space and the 
intraoperative ultrasound image space there is the possibility to 
initially simply determine changes in the area to be operated on by 
means of ultrasound imaging. 

Subsequently, a so-called co-registration between the preoperative 
and the intraoperative images is carried out with the goal to 
transfer this information in real time to the robot arm and to thus 
adjust the planned operation steps to the momentary spatial 
conditions. Moreover, this co-registration step makes it possible 
to maintain a respective qualitatively highest display of the 
momentary conditions with regard to the image representation, which 
serves as an orientation for the surgeon during the intervention, 
in that by means of the co- registration, for example, the 
preoperative 3-D magnetic resonance data set is adjusted and 
displayed according to the spatial changes recognized 
intraoperatively by the ultrasound image. The co-registered 
changed magnetic resonance data set as an artificial product of a 
CO- registration step will thus produce images which have an 
appearance as if, intraoperatively, magnetic resonance imaging had 
been performed. In reality, however, the image information of the 
intraoperative ultrasound imaging has been used to alter the 
preoperative magnetic resonance data set such that the new magnetic 
resonance image corresponds to the actual spatial conditions and is 
identical to the actual ultrasound image. In this way, image 
details of the magnetic resonance image, which cannot be shown in 
the ultrasound image, can optionally also be shown in this modified 
form that is adjusted to the momentary situation so that the 
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surgeon is presented with a high resolution virtual image of the 
area to be operated on. On the other hand, it is also possible to 
represent image contents which do not occur in the pre-operative 
image, i.e., which are obtainable only by means of ultrasound. 

In detail, this co-registration mechanism means that the pre- 
planned target points, distances, surfaces, and spaces of the robot 
space are adjusted respectively to the momentary conditions within 
the area to be operated on. They make possible an updated precise 
operation . 

This means that, for example, when a coagulation probe is activated 
in order to coagulate the central area of a tumor, the entire tumor 
will shrink as a result of the coagulation process and the tumor 
boundaries preset pre-operatively by the surgeon no longer 
correspond to the actual tumor boundaries. The device according to 
the invention is able to observe and to document the movements of 
these preset boundaries, i.e., to register the volume changes and 
to transmit these to the surgical robot. The volume which has been 
predetermined during preoperative planning by the surgeon is 
referred to as the "alarm volume" . The control software for 
registration of changes of this volume or changes of the surface of 
this volume is referred to as the "alarm volume adaptation tool" 
(AVAT) . 

One of the programs for surgical robot performance is designed to 
coagulate automatically a volume pre-planned by the surgeon. In 
this context, the automatic coagulation process is stopped as soon 
as the momentarily coagulated volume has reached at one location 
the planned volume. In this connection, the performance of the 
AVAT must be taken into account which ensures that the planned 
volume toward the end of the coagulation process no longer 
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corresponds to the momentary volume because the entire tumor during 
the coagulation process has shrunk. The coagulation process is 
thus adapted according to the movement of the tumor surface during 
the coagulation process and terminated earlier. 

The next operation step is then comprised of, for example, moving 
the tip of the coagulation probe into the center of that new sphere 
which contains the greatest possible residual tumor proportion to 
be coagulated next. In this connection, the AVAT must be taken 
into account again between planned and momentary volume. Moreover, 
for this and the subsequent steps for the third, fourth etc. 
coagulation, the movement of the probe tip is guided from the 
momentary position to a predetermined entry point and from there 
along an access path to the new target point. 

A further program course is, for example, provided in order to scan 
the inner surface of a cystic tumor with a surgical instrument, to 
first coagulate a predetermined layer thickness and to then scrape 
off the coagulated tissue. In this context, the distance 
maintenance program is activated which ensures that a safety 
distance is maintained relative to the coagulation boundary so that 
bleeding is prevented during the tumor removal , 

The features of the invention disclosed in the above description, 
the claims as well as the drawing can be important individually as 
well as in any chosen combination for the realization of the 
invention in its different embodiments. 
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